Colour Descriptors for Tracking in Spatial Augmented Reality

Augmented Reality is an emerging research field, that aims for the composition of real and virtual imagery, by means of a camera and display device. Spatial augmented reality employs data projectors to augment the real world. In this setting, traditional tracking methods fall short due to the interference caused by the projector. Recent works assume a calibration process to model the projector and assume continuity in movement of the object being tracked. In this paper we present a tracking-by-detection system that does not require such a procedure and makes use of natural features represented by SIFT descriptors. We evaluate a set of photometric invariants that have previously been shown to improve the performance of object recognition, added to the descriptor to reduce the influence of the projector. We evaluate the descriptors based on precision-recall under projector distortion and the total system based on its tracking performance. Results show tracking is significantly more precise using one of the invariants.